Considered broadly, entertainment, whether pre-recorded or live, is designed to be observed from a single perspective. One or more actors (people, animals, machines, animated characters, etc.) are placed into a setting where they convey the story by producing sound, motion, smells, and possibly other sensations. The audience experiences the story by sensing these sensations from an outside, detached, perspective. An exception to the “outside perspective” is first-person video games that allow the user to experience the sensations first hand, from the perspective of the main game character.
For stage productions, the director carefully chooses the set configuration and actor positions that best convey the story to an audience that views the production from a single perspective (except for slight variations due to audience seating position). In such a system, the audience is limited to experiencing the story from a third-person, outside perspective. Furthermore, the audience cannot know the inner thoughts or motivations of the characters in a stage production, since these are necessarily private to the characters and, thus, hidden from the audience.
For film (including video, animation, etc.), the director decides the final audience perspective by selecting a single camera position, though the camera may be moved, using cuts, fades, and other devices, throughout the telling of the story. Unlike live stage productions, the film director has, therefore, the ability to move the observer's viewpoint at any time throughout the production to enhance the immersion of the final viewer. For example, during a conversation between two actors, the camera may constantly cut (switch) between actor perspectives, usually positioning the viewpoint roughly at the position of the listener, so that the speaker appears to be speaking to the viewer. Though these techniques help to immerse the viewer into the story, the viewer is limited, as with stage productions, to experiencing only what can be sensed through sight and sound, and not to the inner thoughts and motivations of the characters.
For first person video games, the virtual camera is usually attached to the main character, so that the viewer maintains the perspective of the main character. For this reason, first person video games may be considered the most immersive of the described forms of entertainment. However, one of the shortcomings of first person perspective is that the story cannot include any elements that are unobserved by the main character. Simultaneous events occurring at spatially distant locations are not known to the viewer. This limitation makes the first person perspective a poor choice for conveying complex story lines with parallel action.
The system and method according to the present disclosure is a multiple viewpoint entertainment system that allows audiences to experience a pre-recorded or live production from any one of multiple perspectives. The viewers are allowed to change available perspective views at any time. The concept of multiple viewpoints is made possible by introducing sensors, such as camera and sound sensing devices, to multiple actors during the production and then delivering the recorded or live results to audience members who have a choice of perspectives.
In the system according to the present disclosure, actors, live or computer-generated, whose viewpoints are to be made available to audience viewers are selected prior to production. Sensor devices, such as a camera and microphone, are attached to or associated with the selected actors or actor positions. When the actor appears in the production, the scene is then captured from that actor's perspective. In cases where multiple actors are in a scene, the scene may be captured from all available perspectives. A “director's view” perspective is a perspective view and is analogous to the previous notion of the film camera.
According to one aspect of the present disclosure, though the sensor devices associated with a given actor would normally be physically attached to the actor, they could also be positioned near to the actor. For example, in cases where the sensor devices would be visible, distracting, or otherwise interfere with the production, the sensor devices could be located proximally to the actor using stand-off sensors that approximate the actor's perspective.
According to another aspect of the present disclosure, in the case of live entertainment such as stage plays, each actor may be equipped with cameras that record that actor's viewpoint, which is then transmitted live to the audience. For recorded productions, the recordings would be stored for post-production using traditional techniques. Each individual actor's recorded views could be optionally enhanced by adding audible “inner thoughts,” adding audio that would only be detected by the actor (for example, from an earphone or headset), processing the sound using effects processors, adding closed-captioning text, or otherwise modifying or augmenting the sensor output. These modifications and additions would be used to further enhance the immersive aspect of the production, and to provide additional information otherwise unknown to the general, third-person, observer.
In the system of the present disclosure, the captured data from each actor would be collected into “perspective view streams” which would then be collected centrally and then made available to audience members, individually or collectively. One method would be to equip each audience member with a virtual reality (VR) device or viewing station that includes video and headphones, enabling the audience member to receive a selected perspective view stream. Another method would be to partition the audience and select a unique perspective stream for each partition. In either method, the stream could be optionally switched dynamically between the available view streams.
According to another aspect of the present disclosure, the above VR device or viewing station would be connected to the viewing system using a wired or wireless connection, allowing real-time transmission of one or more perspective view streams and other information. Wireless techniques would include radio frequency (RF) equipment including a custom transceiver or standard Wireless Local Area Network (WLAN) capabilities (e.g. 802.11n), or WiFi.
Another capability facilitated by the present disclosure would allow the above VR device or viewing station to be optionally equipped with user accessible controls including audio volume and closed captioning for the hearing impaired, video brightness for the visually impaired, alternate language subtitles or another language, buttons or knobs for ratings or other real-time feedback. In cases where all audience members are equipped with a VR device or viewing station, the need for a traditional theater would be eliminated, possibly reducing the cost of such a venue.
According to one embodiment of the present disclosure, optical discs (e.g. DVD and Blu-ray) or digital downloads for home entertainment markets would also be used to store multiple perspective view streams, taking advantage of existing disc/digital capabilities for multiple tracks and camera angles. For home systems possessing the required interfacing capability, the VR device or viewing station would be attachable to such systems, allowing the same in-venue experience in a home setting.
According to another embodiment of the present disclosure, venues could be further augmented to introduce olfactory (smell) stimulation, seat motion, temperature variations, other peripherals, and other sensations.